Heads-up user interface and controls for interactive heads-up display of traffic targets

ABSTRACT

A system for interactive heads-up display (HUD) and control of traffic targets is disclosed. In embodiments, the HUD receives and decodes traffic information from onboard surveillance systems to identify proximate aircraft within a threshold range and the relative positions of each aircraft. The HUD arranges the proximate aircraft into an ordered sequence based on distance from ownship or other priority criteria. When the HUD is active, interactive symbols are displayed at the relative positions of the proximate aircraft (e.g., or at the edges of the display for air traffic within range but outside the HUD field of view). A heads-up controller is operable by the pilot and allows the pilot to highlight each interactive symbol in sequence with a displayable cursor, selecting and designating the associated proximate aircraft for spacing or traffic applications without taking pilot focus off the HUD.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to, and incorporates by reference in itsentirety, the following U.S. Patent Applications:

Concurrently filed U.S. patent application Ser. No. XX/XXX,XXX, entitledMETHOD FOR INTERACTIVE HEADS-UP DISPLAY AND CONTROL OF TRAFFIC TARGETSand having docket number 126278US01; and

Concurrently filed U.S. patent application Ser. No. XX/XXX,XXX, entitledHEADS-UP USER INTERFACE AND VOICE ACTIVATED CONTROLS FOR DISPLAY OFTRAFFIC TARGETS and having docket number 126280US02.

BACKGROUND

Avionic devices that provide a cockpit display of traffic information(CDTI) capability will enable pilots to supplement traffic reports andcommunications from air traffic control (ATC) and proactively maintainseparation from proximate aircraft. For example, CDTI may provide visualrepresentations of a traffic environment, identifying proximate aircraft(e.g., along with the relative positions and/or intent of saidaircraft).

CDTI is conventionally displayed in navigation-map format (e.g.,overhead view) on a heads-down display (HDD). As a result, when usingCDTI or related traffic applications (e.g., CDTI Assisted VisualSeparation (CAVS)), the pilot must go heads-down to confirm the locationof a traffic target (or, e.g., to designate the target for trafficapplications). Conventional systems that incorporate a heads-up display(HUD) generally do not display traffic information via the HUD to reducevisual clutter.

SUMMARY

A system for interactive heads-up display and control of traffic targetsis disclosed. In embodiments, the system comprises a heads-up display(HUD) and a heads-up controller operable by the pilot/co-pilot withoutlooking away from or otherwise taking focus off the HUD. In embodiments,the HUD receives and decodes traffic information from onboard trafficcollision avoidance systems (TCAS), automatic dependentsurveillance-broadcast (ADS-B), or other surveillance systems toidentify proximate aircraft within threshold range and their relativepositions. The HUD arranges the proximate aircraft in order of priority,e.g., in an ordered sequence based on the distance of each proximateaircraft or other desired criteria. The HUD activates traffic overlay bydisplaying interactive symbols over each proximate aircraft based ontheir relative positions (or, e.g., at the edges of the display for“parked” aircraft within range but outside the field of view of theHUD). The heads-up controller allows the pilot to activate or deactivatetraffic overlay. In embodiments, the heads-up controller allows thepilot to highlight each proximate aircraft in sequence via itsinteractive symbol by traversing a displayable cursor through theordered sequence (e.g., in order of priority). The heads-up controllerallows the pilot to select a highlighted aircraft and designate aselected aircraft for spacing or traffic applications.

In some embodiments, the interactive traffic symbols are of a first type(e.g., solid lines) for proximate aircraft within the field of view ofthe HUD and of a second type (e.g., broken lines) for “parked” aircraftoutside the field of view and represented by interactive traffic symbolsplaced at the edges of the display.

In some embodiments, the HUD displays a tail number or other uniqueidentifier adjacent to the interactive symbol for each proximateaircraft.

In some embodiments, the HUD displays additional traffic informationadjacent to the interactive traffic symbols for selected or designatedproximate aircraft, e.g., relative position information, airspeedinformation, trajectory information (e.g., climbing/descending), spacinginformation.

In some embodiments, the additional traffic information is presented viaa dynamic text display.

In some embodiments, the HUD displays interactive traffic symbols withincreased prominence (e.g., larger, brighter) based on a higher priorityof the corresponding aircraft within the ordered sequence (e.g., closerto the ownship).

In some embodiments, the HUD displays interactive traffic symbols withreduced prominence (e.g., smaller, dimmer) based on a lower priority ofthe corresponding aircraft within the ordered sequence (e.g., moredistant from the ownship).

In some embodiments, the HUD superimposes interactive traffic symbolsover other displayed content (e.g., synthetic vision systems, enhancedvision systems, combined vision systems).

In some embodiments, the HUD automatically discontinues the display ofinteractive traffic symbols if an unusual attitude of the aircraft isdetected.

In some embodiments, the HUD automatically discontinues the display ofinteractive traffic symbols in the event of a resolution advisory (RA)issued by an onboard traffic collision avoidance system (TCAS).

In some embodiments, the HUD detects a data loss (e.g., a reportingaircraft fails to update position information) or data fault (e.g.,position information is outside accuracy or data integrity guidelines),and the HUD removes the interactive traffic symbols for any aircraftassociated with the data loss or data fault.

In some embodiments, the HUD includes a head-worn or helmet-mounteddisplay (HWD, HMD) having a dynamic field of view aligned with theorientation of the pilot/co-pilot.

This Summary is provided solely as an introduction to subject matterthat is fully described in the Detailed Description and Drawings. TheSummary should not be considered to describe essential features nor beused to determine the scope of the Claims. Moreover, it is to beunderstood that both the foregoing Summary and the following DetailedDescription are example and explanatory only and are not necessarilyrestrictive of the subject matter claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. The use of the same reference numbers in different instances inthe description and the figures may indicate similar or identical items.Various embodiments or examples (“examples”) of the present disclosureare disclosed in the following detailed description and the accompanyingdrawings. The drawings are not necessarily to scale. In general,operations of disclosed processes may be performed in an arbitraryorder, unless otherwise provided in the claims. In the drawings:

FIG. 1 is an illustration of an interactive heads-up display (HUDaccording to example embodiments of this disclosure;

FIG. 2A is a block diagram of the interactive HUD of FIG. 1;

FIG. 2B illustrates operations of the interactive HUD of FIG. 1;

FIG. 3 is an illustration of example symbology of the interactive HUD ofFIG. 1;

FIG. 4 is an illustration of example symbology of the interactive HUD ofFIG. 1;

FIGS. 5A through 5E are flow diagrams illustrating a method forinteractive display of traffic targets via a HUD according to exampleembodiments of this disclosure;

FIG. 6A is a block diagram of the HUD of FIG. 1 incorporating a heads-upmanual control system;

FIG. 6B is a diagrammatic illustration of example operations of the HUDof FIG. 6A;

FIG. 7A is a block diagram of the HUD of FIG. 1 or 6A incorporating aheads-up voice-activated command system;

and FIGS. 7B through 7D are diagrammatic illustrations of exampleoperations of the HUD of FIG. 7A.

DETAILED DESCRIPTION

Before explaining one or more embodiments of the disclosure in detail,it is to be understood that the embodiments are not limited in theirapplication to the details of construction and the arrangement of thecomponents or steps or methodologies set forth in the followingdescription or illustrated in the drawings. In the following detaileddescription of embodiments, numerous specific details may be set forthin order to provide a more thorough understanding of the disclosure.However, it will be apparent to one of ordinary skill in the art havingthe benefit of the instant disclosure that the embodiments disclosedherein may be practiced without some of these specific details. In otherinstances, well-known features may not be described in detail to avoidunnecessarily complicating the instant disclosure.

As used herein a letter following a reference numeral is intended toreference an embodiment of the feature or element that may be similar,but not necessarily identical, to a previously described element orfeature bearing the same reference numeral (e.g., 1, 1a, 1b). Suchshorthand notations are used for purposes of convenience only and shouldnot be construed to limit the disclosure in any way unless expresslystated to the contrary.

Further, unless expressly stated to the contrary, “or” refers to aninclusive or and not to an exclusive or. For example, a condition A or Bis satisfied by any one of the following: A is true (or present) and Bis false (or not present), A is false (or not present) and B is true (orpresent), and both A and B are true (or present).

In addition, use of “a” or “an” may be employed to describe elements andcomponents of embodiments disclosed herein. This is done merely forconvenience and “a” and “an” are intended to include “one” or “at leastone,” and the singular also includes the plural unless it is obviousthat it is meant otherwise.

Finally, as used herein any reference to “one embodiment” or “someembodiments” means that a particular element, feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment disclosed herein. The appearances of thephrase “in some embodiments” in various places in the specification arenot necessarily all referring to the same embodiment, and embodimentsmay include one or more of the features expressly described orinherently present herein, or any combination or sub-combination of twoor more such features, along with any other features which may notnecessarily be expressly described or inherently present in the instantdisclosure.

Broadly speaking, embodiments of the inventive concepts disclosed hereinare directed to a heads-up display (HUD) and user interfaceincorporating interactive heads-up display and control of traffictargets. For example, traffic information may be displayed via the HUDin conformance to actual proximate traffic, and interactive controlsallow the pilot to directly manage traffic targets through the HUD whileeyes-out, eliminating the need to cycle back and forth with heads-downtraffic displays and enhancing general situational awareness.

Referring to FIG. 1, a heads-up display 100 (HUD) incorporating heads-upinteractive traffic overlay is disclosed.

In embodiments, the HUD 100 may include a fixed transparent displaypositioned in a line of sight between the eyes/head of thepilot/co-pilot and a forward cockpit window. In some embodiments, theHUD 100 may include head-worn or helmet-mounted displays (HWD, HMD) wornby a pilot/co-pilot or operator of the aircraft and presenting a dynamicview based on the orientation of the pilot's head relative to theaircraft. In some embodiments the HUD 100 may include one or moresynthetic display feeds corresponding to a field of view (FOV). Forexample, the FOV may be a static FOV fixed relative to the aircraft(e.g., in the case of a HUD fixed in front of an aircraft window), andcorresponding to the display size, e.g., X degrees horizontal by Ydegrees vertical. Alternatively, the FOV may be a dynamic FOV, e.g., theFOV of HMD/HWD systems may be fixed to the position and orientation(e.g., pose) of the head of the pilot or operator, and may shift as thehead moves.

In embodiments, the HUD 100 may incorporate Synthetic Vision Systems(SVS), Enhanced Vision Systems (EVS), Combined Vision Systems (CVS), andother like means of visually representing the environment proximate tothe aircraft. For example, Combined Vision displays may includereal-time camera feeds (e.g., visible light and/or infrared) of theterrain 102 over which the aircraft is in flight, enhanced by terraindata or other supplementary information. The HUD 100 may include aprimary flight display (PFD) or selected PFD instruments or components,e.g., artificial horizon 104, airspeed indicator 106, altimeter 108,heading indicator 110.

In some embodiments, the HUD 100 may include proximate air traffic 112.For example, proximate air traffic 112 may include any commercial orcivil aircraft close enough to the ownship for visual identification(e.g., and any unmanned aircraft close enough, or large enough, forvisual identification). For example, while the pilot may be able toconfirm the presence of proximate air traffic 112, e.g., via visualidentification, the pilot may not be able to confirm the uniqueidentities of each proximate aircraft 112 via visual identificationalone. Accordingly, conventional methods may require the pilot toalternate between a HUD and a heads-down Cockpit Display of TrafficInformation (CDTI) or other traffic display in order to match eachvisually identified proximate aircraft to a traffic target displayed viaCDTI.

In embodiments, the HUD 100 may receive CDTI traffic information andgenerate a HUD traffic overlay comprising interactive symbols matched toeach proximate aircraft 112 according to its relative position, e.g.,where the proximate aircraft would appear as viewed through thetransparent display surface. For example, the aircraft may be equippedwith collision avoidance (e.g., aircraft collision avoidance systems,traffic collision avoidance systems (ACAS, TCAS)) and/or surveillanceradar systems (including, but not limited to, automatic dependentsurveillance-broadcast (ADS-B) or other sensor-based systems capable ofreceiving encoded messages from ground-based air traffic controlfacilities or directly from proximate aircraft. Traffic information mayinclude identifying information (e.g., tail numbers or other uniqueidentifiers) and position information (e.g., latitude, longitude,altitude) of any adequately configured aircraft in range. Trafficinformation may include unmanned aircraft systems (UAS) if said UAS areconfigured with the proper equipment for ownship position determinationand ADS-B information. In embodiments, the HUD 100 traffic overlay maybe superimposed over other visual content presented by the HUD (e.g.,SVS/EVS/CVS feeds), or the HUD traffic overlay may be displayedindependent of any other visual content.

In embodiments, all aircraft corresponding to a position and identifierreceived by the HUD 100 may be displayed by the HUD as interactivesymbology 114 corresponding to the decoded position information. Forexample, all identified aircraft within a threshold range and within thefield of view of the HUD 100 may be represented by interactive symbology114 based on the relative position of the identified aircraft to theownship position of the embodying aircraft, e.g., as determined byonboard global navigation satellite system (GNSS)-based or other likeposition receivers.

In embodiments, the interactive symbology 114 of the traffic overlaysystem of the HUD 100 may indicate not only the relative position of allproximate aircraft 112 but also a status of each proximate aircraftrelative to the HUD traffic overlay system. For example, the interactivesymbology 114 may additionally indicate if a proximate aircraft 112 iscurrently highlighted by the HUD 100 (e.g., via a cursor 116), selectedby the HUD, or designated by the HUD (e.g., as an “othership”) for CDTIassisted visual separation (CAVS), Flight Deck Interval Management(FIM), and/or other like approach spacing or traffic applications. Insome embodiments, traffic information may include proximate aircraft 112whose positions are within range of the HUD 100 traffic overlay systembut whose positions relative to the ownship position are outside thefield of view of the HUD 100. Interactive symbology 114 a may indicatesuch “parked” aircraft whose relative positions are outside the field ofview, e.g., behind the aircraft (or behind the current orientation ofthe pilot, for HMD/HWD implementations). For example, “parked” aircraftmay be represented by interactive symbology 114 a incorporating brokenor dotted lines and positioned along an edge of the FOV of the HUD 100corresponding to the relative position of the “parked” aircraft (e.g.,at an approximately 7 o'clock bearing relative to the ownship).

In embodiments, the HUD 100 may display each interactive symbol 114, 114a such that the center of the displayed symbol corresponds with thereported position and altitude data of the corresponding proximateaircraft 112. For example, if received position and altitude data is notwithin predetermined accuracy tolerances, interactive symbology may notbe displayed. In some embodiments, the size of a displayed symbol may befixed or may be dynamic based on one or more preselected factors, e.g.,distance/range and/or position accuracy/uncertainty.

In embodiments, the HUD 100 may include an interactive heads-up controlsystem allowing the pilot to manage traffic target display andapplications without looking away from the HUD. For example, theheads-up controls allow the pilot to toggle HUD 100 traffic overlaybetween active and inactive status (e.g., “on” or “off”) andcycle/traverse a cursor through each identified proximate aircraft 112in sequence. All proximate aircraft 112 may be organized into an orderedsequence or hierarchy by the HUD 100 based on one or more criteriaselected by the pilot; for example, sequence criteria may be preset inadvance or modified via the heads-up control, as disclosed in greaterdetail below. The interactive symbols 114 corresponding to eachproximate aircraft 112 may vary to indicate the current status of eachproximate aircraft with respect to the HUD 100 traffic overlay and withrespect to CAVS and other traffic applications, including (but notlimited to): normal; highlighted (e.g., a proximate aircraft currentlyhighlighted by the cursor 116, but which the pilot has not selected ordesignated); selected (e.g., a proximate aircraft which the pilot hasselected for possible traffic applications; may be highlighted ornon-highlighted, e.g., the cursor has been moved or has timed out);designated (e.g., designated by the pilot for CAVS or another trafficapplication).

Referring now to FIGS. 2A and 2B, the HUD 100 traffic overlay system isdisclosed. The HUD 100 may include control processors 202, heads-updisplay components 204 (e.g., including graphics processors and displaysurfaces), and heads-up controller 206.

In embodiments, the control processors 202 may receive trafficinformation from traffic collision avoidance systems 208 (TCAS), ADS-BIn receivers 210, and other like collision avoidance and/or trafficmonitoring systems. For example, traffic information may include uniqueidentifiers (e.g., tail numbers, ICAO identifiers) and positioninformation (latitude, longitude, altitude) corresponding to proximateaircraft (112, FIG. 1). Based on received traffic information, thecontrol processors 202 may display interactive symbology 212 a-ecorresponding to the positions of reporting proximate aircraft 112 (or,e.g., superimpose interactive symbology over the synthetic displayimagery (e.g., EVS, SVS)), as shown by FIG. 2B. (See also, e.g., FIG. 1,114/114 a.)

Referring also to FIG. 2B, in embodiments the HUD 100 traffic overlaymay arrange and/or modify interactive symbology 212 a-e based on one ormore criteria (e.g., horizontal distance from ownship, angulardisplacement from boresight) selectable by the pilot and modifiable viathe heads-up controller 206. For example, the control processors 202 maydetermine that the position of the proximate aircraft corresponding tothe interactive symbol 212 a is closest to the ownship position, andthat therefore the interactive symbol 212 a is to be placed first in theordered sequence of interactive symbols 212 a-d. In some embodiments,the interactive symbol 212 a may also be displayed with increased orreduced prominence (e.g., greater or lesser brightness, increased ordecreased size, change in color) relative to other interactive symbolsto reflect the proximity of the corresponding aircraft 112 (e.g., ormore generally the priority of the corresponding aircraft with respectto the ordered sequence of aircraft reporting position information).Similarly, the pilot may (e.g., via the heads-up controller 206, advancein turn through the interactive symbols 212 b-d corresponding to moredistant aircraft 112 (e.g., but still within the FOV of the HUD 100),and then to the interactive symbol 212 e corresponding to a “parked”aircraft behind the ownship and/or its pilot (and thereby represented bya dashed symbol positioned at an edge of the FOV of the HUD 100 andcorresponding to the relative position of the “parked” aircraft).

In some embodiments, the control processors 202 may be in communicationwith attitude sensors or other aircraft sensors. For example, if anunusual attitude of the aircraft is detected, the HUD 100 mayautomatically discontinue interactive symbology 212 a-e to reduceclutter within the HUD (which may distract the pilot from resolving theunusual attitude). Similarly, the TCAS 208 may issue a resolutionadvisory (RA) to the pilot or flight crew (e.g., recommending maneuversand/or maneuver restrictions to either achieve or maintain separationfrom a threat). In some embodiments, the HUD 100 may automaticallydiscontinue interactive symbology 114, 212 a-e in the event of an RAissued by the TCAS 208. In some embodiments, the HUD 100 may remove oneor more interactive symbols 212 a-e in the event of a data loss or datafault with respect to the position information reported by thecorresponding proximate aircraft 112. For example, if a proximateaircraft 112 ceases transmitting position information, or thetransmission is not received, or position information is not within adesired accuracy tolerance, the corresponding interactive symbol 212 a-emay be removed from the HUD 100 traffic overlay until accurate positioninformation is once again received.

Referring now to FIG. 3, the interactive symbology 114, 114 a of the HUDtraffic 100 overlay of FIG. 1 is disclosed.

In embodiments, the interactive symbology 114 may indicate a status(e.g., with respect to CDTI and any applicable traffic applications) ofeach proximate aircraft (112, FIG. 1) for which the HUD 100 has trafficinformation (e.g., an identifier and position). For example, the symbol302 may indicate a proximate aircraft 112 having normal status, e.g.,any aircraft within the threshold range, within the field of view, andnot otherwise highlighted, selected, or designated. The symbol 304 mayindicate a proximate aircraft 112 having normal status (e.g., neitherselected nor designated) but which is currently highlighted by the HUD100 traffic overlay cursor 116. The symbol 306 may indicate a proximateaircraft 112 selected by the HUD 100 traffic overlay and which is stillhighlighted by the cursor 116. For example, the pilot (e.g., via theheads-up controller 206) may select a traffic target for display ofadditional received or derived traffic information (as disclosed ingreater detail below) but without designating the traffic for CAVS orother like applications. Similarly, the symbol 308 may indicate aproximate aircraft 112 selected but no longer highlighted by the cursor116. The symbol 310 may indicate a proximate aircraft 112 designated forCAVS by the pilot. For example, when a traffic target is selected (306),the HUD 100 traffic overlay may display a menu of traffic applicationsfrom which the pilot may select a specific application for which thecorresponding proximate aircraft will be designated as “othership”. Theproximate aircraft may shift status from selected/highlighted (306) todesignated (310), as the cursor 116 is removed upon designation.

In embodiments, the interactive symbology 114 a, e.g., the interactivesymbols 312, 314, 316, 318, 320, may be implemented and may functionsimilarly to the interactive symbols 302, 304, 306, 308, 310, exceptthat the interactive symbols 312, 314, 316, 318, 320 may indicate“parked” aircraft having, respectively, a normal status (312);highlighted/deselected status (314); highlighted/selected status (316);selected/non-highlighted status (318); and designated status (320). Forexample, the interactive symbols 312, 314, 316, 318, 320 may bedisplayed by the HUD 100 traffic overlay similarly to the interactivesymbols 302, 304, 306, 308, 310, except that the interactive symbols312, 314, 316, 318, 320 may be fully or partially rendered in dashed orbroken lines.

In some embodiments, the heads-up controller 206 may include a controlknob or other like heads-up manual controller 206 a manually operable bythe pilot/co-pilot without looking away from, or otherwise taking theirfocus off, the HUD 100. In some embodiments, the heads-up controller 206may include a voice-activated command system 206 b; for example, thepilot may manually trigger the voice-activated command system to listenfor and decode verbal command input and perform HUD 100 traffic overlayfunctions based on the decoded command input. Heads-up controllersystems 206 a-b are disclosed in greater detail below. In embodiments,the interactive symbology 114, 114 a is not limited to the symbolsdescribed above, but may include other symbols and combinations ofsymbols configured to convey the same information.

Referring to FIG. 4, in embodiments, the HUD (100, FIG. 1) trafficoverlay may augment interactive symbology (114/114 a, FIG. 3) bydisplaying additional information corresponding to the proximateaircraft (112, FIG. 1) associated with a displayed interactive symbol.For example, the interactive symbols 302 and 304 may correspond to aproximate aircraft 112 having tail number N60RC (e.g., or some otherunique identifier corresponding to the particular aircraft), whichinformation may be obtained by the HUD 100 traffic overlay by decodinginbound position messages received by a TCAS or ADS-B receiver (208,210; FIG. 2). The tail number or identifier (402) may be displayeddirectly below the corresponding interactive symbols 302, 304 to aid invisual confirmation of the specific and correct traffic target.

In embodiments, the interactive symbol 308 a may be implemented and mayfunction similarly to the interactive symbol 308 of FIG. 3, except thatfor selected and/or designated traffic targets (e.g., also includingselected/highlighted traffic targets (306, FIG. 3) and designatedtraffic targets (310, FIG. 3)), the HUD 100 traffic overlay may displayadditional textual information and/or graphic information. For example,the control processors (202, FIG. 2) may derive additional pertinentinformation about specific proximate aircraft (112, FIG. 1) based onposition information received from the proximate aircraft (or, e.g.,based on trends observed within multiple position reports over time fromthe same traffic target). For example, the HUD 100 traffic overlay maydisplay textual traffic information as a single-line display cyclingbetween multiple data points (e.g., distance to the traffic target,relative to the ownship position (“3.5 NM”, 404 a); traffic speed(absolute or relative) (“230 KTS”, 404 b); traffic spacing/intervaldistance for a set time interval (e.g., 2 seconds per data point);category (e.g., single-engine land, rotorcraft, powered-lift, glider, orother aviation class rating.) Further, the HUD 100 traffic overlay maydisplay relative altitude (e.g., of the traffic target relative to theownship; plus signs (+) indicate altitude above and minus signs (−)indicate altitude below) (“+1000 FT”, 404 c). The HUD 100 trafficoverlay display may further include an upward-pointing climb arrow (406)or downward-pointing descent arrow based on whether the proximateaircraft 112 is climbing or descending.

Referring to FIG. 5A, the method 500 may be implemented by the HUD 100and may include the following steps.

At a step 502, the HUD receives traffic information associated with oneor more proximate aircraft within a threshold range of the aircraft. Forexample, the traffic information may be received and decoded by onboardcollision avoidance and/or surveillance systems (e.g., traffic collisionavoidance system (TCAS), automatic dependent surveillance-broadcast(ADS-B) receiver). In some embodiments, the traffic information mayinclude aircraft within the field of view (FOV) of the HUD as well asaircraft within the threshold range but outside the FOV.

At a step 504, the HUD traffic overlay organizes and arranges allproximate aircraft (e.g., all proximate aircraft for which trafficinformation has been received) into an ordered sequence according to oneor more selectable proximity criteria, e.g., distance from the ownship(horizontal and/or vertical).

At a step 506, the HUD displays interactive symbols over each proximateaircraft within the threshold range and associated with received trafficinformation. For example, the interactive symbols may indicate thestatus of the proximate aircraft relative to the HUD (e.g.,default/normal; highlighted; selected; designated for trafficapplications). In some embodiments, interactive symbols may be displayedat increased or reduced prominence (e.g., greater or lesser brightness,greater or lesser size) to indicate higher or lower priority within theordered sequence, e.g., closer aircraft may be associated withlarger/brighter symbols and more distant aircraft with smaller/dimmersymbols. In some embodiments, interactive symbols for proximate aircraftmay include tail numbers or other unique aircraft identifiers. In someembodiments, interactive symbols for selected and designated aircraftmay additionally include other received or derived traffic information,e.g., a distance or airspeed of the proximate aircraft; whether theproximate aircraft is above or below the ownship and whether it isclimbing or descending. The HUD may display the interactive symbolsindependently or superimposed over other displayed visual feeds, e.g.,SVS/EVS/CVS.

At a step 508, the HUD traffic overlay accepts control input from apilot/co-pilot, operator, or user via a heads-up controller. Forexample, the heads-up controller may include a control knob operable bythe pilot while focused on the HUD, or a voice-activated controller.

Referring now to FIG. 5B, a step 510, the HUD traffic overlay maytransition, in turn, each interactive symbol corresponding to aproximate aircraft from a non-highlighted to a highlighted status bytraversing the ordered sequence based on the received control input. Forexample, the HUD traffic overlay may highlight with a cursor eachinteractive symbol in turn as directed by the user in decreasing orderof distance or priority (or, e.g., based on other selectable prioritycriteria). In some embodiments, the HUD traffic overlay may display,next to any interactive symbol corresponding to a proximate aircrafthaving a normal/default status (highlighted or unhighlighted), a tailnumber or other unique identifier of the aircraft.

At a step 512, the HUD traffic overlay transitions a cursor-highlightedinteractive symbol (e.g., and its corresponding aircraft) to a selectedstatus based on the accepted control input. For example, when aproximate aircraft is selected by the HUD traffic overlay, the trafficoverlay may display additional content associated with, e.g., therelative position, airspeed, spacing, category, and/or trajectory of theproximate aircraft.

At a step 514, the HUD traffic overlay designates a selected proximateaircraft for one or more traffic applications (e.g., as an “othership”for CAVS or spacing purposes) based on the accepted control input (e.g.,and adjusting the corresponding interactive symbol accordingly). Forexample, the cursor may be removed from the displayeddesignated-aircraft symbol.

Referring now to FIG. 5C, the method 500 may include an additional step516. At the step 516, the HUD may automatically discontinue the trafficoverlay to reduce clutter if an unusual attitude of the ownship isdetected (e.g., via onboard attitude sensors).

Referring now to FIG. 5D, the method 500 may include an additional step518. At the step 518, the HUD may automatically discontinue the trafficoverlay to reduce clutter. if the onboard TCAS issues a resolutionadvisory (RA).

Referring now to FIG. 5E, the method 500 may include additional steps520 and 522. At the step 520, the HUD may detect a data loss or datafault with respect to received position information. For example,position information transmissions from one or more proximate aircraftmay not be received, or the received position information may beinvalid, e.g., outside prescribed accuracy or data integrity limits.

At the step 522, the HUD removes the interactive symbols for anyproximate aircraft associated with lost, invalid, or otherwise faultyposition information.

Referring to FIG. 6A, the HUD 100 a may be implemented and may functionsimilarly to the HUD 100 of FIGS. 1 through 4, except that the HUD 100 amay incorporate a heads-up manual control knob 206 a.

In embodiments, the heads-up manual control knob 206 a may be configuredfor controlled rotation 602 and downward articulation 604 (e.g.,press-and-release). For example, controlled rotation 602 may involverotating (e.g., tabbing) the heads-up control knob 206 a in a partialarc, either clockwise or counterclockwise. The heads-up manual controlknob 206 a may be spring-loaded or otherwise configured to retract (606)back to its initial position after being rotated 602 in eitherdirection.

Referring also to FIG. 6B, example operations of the heads-up manualcontrol knob 206 a and the HUD 100 a are disclosed. In embodiments, theheads-up manual control knob 206 a may incorporate combinations ofcontrolled rotation 602 and downward articulation 604 to allow thepilot/co-pilot to convey control input to the HUD 100 a without lookingaway from, or otherwise shifting focus from, the HUD.

For example, the pilot may press down (604 a) to activate the HUD 100 atraffic overlay (608; similarly, pressing down may deactivate the HUDtraffic overlay if is currently active). The activated traffic overlaymay display interactive traffic symbols (114, FIG. 1) for all proximateaircraft (112, FIG. 1) within the threshold range reporting positioninformation. To activate the cursor (116, FIG. 1) the pilot may pressdown again (604 b) to highlight the highest priority traffic target(610, e.g., the proximate aircraft 112 closest to the ownship). Tohighlight a different traffic target, the pilot may rotate (602 a) theheads-up manual control knob 206 a counterclockwise or clockwise tosequentially traverse each interactive traffic symbol 114 in ascendingor descending sequential order of priority (612).

Once the desired traffic target is highlighted, the pilot may press down(604 c) to select the highlighted target (614), and press down once more(604 d) to open an application dialog for the selected target (616). Byrotating (602 b), the pilot may shift control from the cursor 116 to theapplication dialog (618), and by rotating again (602 c) the pilot mayscroll through available spacing or traffic applications in theapplication dialog to designate a specific application (e.g., CAVS) foruse with the selected target (620). By pressing down (604 e), anapplication exit dialog may be opened (622), and by pressing down oncemore (604 f), the application dialog is exited and the selected traffictarget designated for the desired application (624), whereby the cursor116 disappears from the now-designated traffic target.

In some embodiments, the heads-up controller 206 may include, but is notlimited to: existing flight deck controls; multifunction dual stack knob(DSK); data tabber knob on a cursor control panel (CCP); ormultifunction keypad (MKP).

Referring now to FIG. 7A, the HUD 100 b may be implemented and mayfunction similarly to the HUDs 100, 100 a of FIGS. 1 through 6B, exceptthat the HUD 100 b may incorporate a voice-activated command system 206b. The voice-activated command system may include a press-and-releasecontrol knob 702 and a microphone 704 for receiving vocal control inputfrom the pilot/co-pilot. In embodiments, the HUD 100 b may be embodiedin a head-worn or helmet-mounted display (HWD, HMD) system, and themicrophone 704 may be similarly head-worn or helmet-mounted.

In embodiments, the voice-activated command system 206 b may be usedinstead of, or in conjunction with, the heads-up manual control knob 206a of FIGS. 6A-B. For example, the HUD 100 b may be trained via machinelearning techniques to decode or parse vocal control input spoken by thepilot and identify within the decoded control input command instructions(e.g., specific words or phrases recognizable by the HUD and associatedwith specific executable instructions, operations, or commandsequences).

In embodiments, the pilot may hold down (706) the press-and-releasecontrol knob 702 to indicate vocal control input spoken into themicrophone 704, which vocal control input may be converted into digitalsignals and analyzed by the control processors 202. For example, thecontrol processors 202 may be trained (e.g., via analysis of the pilot'svoice) to identify with sufficient confidence particular words andphrases spoken by the pilot, parsing the words and phrases intoexecutable instructions. When the pilot has finished speaking, releasingthe control knob 702 may signal the HUD 100 b to cease “listening” viathe microphone 704.

Referring generally to FIGS. 7B through 7D, example operations of thevoice-activated command system 206 b are disclosed.

In embodiments, referring in particular to FIG. 7B, the pilot mayactivate the HUD 100 b traffic overlay (708) by quickly pressing andreleasing (710) the control knob 702 (or, e.g., deactivate the HUDtraffic overlay if it is currently active). In some embodiments,activation of the HUD 100 b traffic overlay (708) may also bevoice-activated. For example, as an alternative to pressing andreleasing (710) the control knob 702, the pilot may activate the HUD 100b traffic overlay (708) by speaking a predetermined command word orphrase (710 a; e.g., “WAKEUP”) followed by a specific traffic overlaycommand (e.g., “TRAFFIC ELEVEN O'CLOCK THREE THOUSAND” (712)).Similarly, in some embodiments the HUD 100 b traffic overlay may beprepared for spoken traffic overlay commands (e.g., as described below)by spoken command phrases 710 a (e.g., “WAKEUP”, “READY”) as analternative to holding down (706 a) the control knob 702.

In embodiments, the pilot may verbally highlight or select traffictargets based on the position of identified proximate traffic targetsproviding traffic information (e.g., proximate aircraft (112, FIG. 1)identified by the HUD 100 b with interactive traffic symbols (114,FIG. 1) displayed by the HUD. For example, given an active HUD 100 btraffic overlay (708) the pilot may hold down (706 a) the control knob702, say, e.g., “TRAFFIC ELEVEN O'CLOCK THREE THOUSAND” (712), andrelease (714 a) the control knob, directing the HUD 100 b to highlight atraffic target, e.g., an interactive traffic symbol (114, FIG. 1)corresponding to a proximate aircraft at a heading around 330 degreesrelative to the ownship (e.g., ahead and slightly to port) and at analtitude around 3,000 ft MSL. If no air traffic is found at or near thisposition, the HUD 100 b may so indicate via visual or aural alert, e.g.,“NO TRAFFIC TARGETS AVAILABLE”. If a traffic target is found at or nearthe indicated position, the HUD 100 b may highlight the interactivetraffic symbol 114 corresponding to the traffic target with a displayedcursor (116, FIG. 1).

In embodiments, the HUD 100 b may identify multiple traffic targets ator near the location indicated by the pilot, or may highlight a traffictarget other than the desired traffic target (716). By holding down (706b) the control knob 702, saying, e.g., “CYCLE LEFT”/CYCLE RIGHT” or“NEXT TARGET”/“PREVIOUS TARGET” (718), and releasing the control knob(714 b), the pilot may scroll through identified traffic targets inascending or descending order of priority until the desired traffictarget is highlighted (720). By holding down (706 c) the control knob702, saying, e.g., “DESIGNATE TRAFFIC FOR CAVS [e.g., or another desiredspacing or traffic application]” (722), and releasing the control knob(714 c), the highlighted traffic target may be designated for thedesired application (724). Similarly, to terminate a designatedapplication, the pilot may hold down (706 d) the control knob 702, say,e.g., “CANCEL [e.g., END/TERMINATE] CAVS” (726), and releasing thecontrol knob (714 d), terminating the application (728).

In embodiments, referring also to FIG. 7C, the voice-activated commandsystem 206 b may allow the pilot to designate a specific traffic targetfor spacing or traffic applications based on the tail number oridentifier (402, FIG. 4) provided by the proximate aircraft 112 anddisplayed adjacent to the interactive traffic symbol 114 for theproximate aircraft. For example, the pilot may hold down (706 e) thecontrol knob 702 (or provide spoken command phrases 710 a), say“DESIGNATE NOVEMBER-SIX-ZERO-ROMEO-CHARLIE FOR CAVS [or other desiredapplication]” (730), and release (714 e) the control knob. The HUD 100 bmay identify the traffic target corresponding to the proximate aircraft112 having tail number N60RC (402) and designate the aircraft (if saidaircraft is found) for traffic applications (732).

In embodiments, referring also to FIG. 7D, the voice-activated commandsystem 206 b may allow the pilot to set or update other HUD or trafficoverlay parameters. For example, the pilot may hold down (706 f) thecontrol knob 702 (or provide spoken command phrases 710 a), say “SETRANGE ALERT THRESHOLD THREE MILES (734), and release (714 f) the controlknob, updating the threshold range to 3 NM and displaying the updatedrange (736) via the HUD 100 b. By pressing and releasing (710) thecontrol knob 702, the updated threshold range may be confirmed by theHUD 100 b (738).

CONCLUSION

It is to be understood that embodiments of the methods disclosed hereinmay include one or more of the steps described herein. Further, suchsteps may be carried out in any desired order and two or more of thesteps may be carried out simultaneously with one another. Two or more ofthe steps disclosed herein may be combined in a single step, and in someembodiments, one or more of the steps may be carried out as two or moresub-steps. Further, other steps or sub-steps may be carried out inaddition to, or as substitutes to one or more of the steps disclosedherein.

Although inventive concepts have been described with reference to theembodiments illustrated in the attached drawing figures, equivalents maybe employed and substitutions made herein without departing from thescope of the claims. Components illustrated and described herein aremerely examples of a system/device and components that may be used toimplement embodiments of the inventive concepts and may be replaced withother devices and components without departing from the scope of theclaims. Furthermore, any dimensions, degrees, and/or numerical rangesprovided herein are to be understood as non-limiting examples unlessotherwise specified in the claims.

1. A system for interactive heads-up display and control of traffictargets, comprising: an aircraft-based heads-up display (HUD) configuredto display a field of view (FOV) proximate to an aircraft; controlprocessors in communication with the HUD and configured to: receivetraffic information from at least one surveillance system of theaircraft, the traffic information associated with one or more proximateaircraft within a threshold range of the aircraft, the one or moreproximate aircraft comprising 1) at least one first proximate aircraftwithin the FOV and 2) at least one parked aircraft outside the FOV;arrange the one or more proximate aircraft into at least one orderedsequence; and display a traffic overlay comprising one or moreinteractive traffic symbols via the HUD, the one or more interactivetraffic symbols corresponding to the one or more proximate aircraft andeach interactive traffic symbol indicating a status of the correspondingproximate aircraft; and a heads-up controller operable by an operator toprovide control input to the control processors, the control inputconfigured for directing the HUD to: automatically transition thetraffic overlay from an active state to an inactive state based on atleast one of: an unusual attitude detected by at least one attitudesensor of the aircraft; or a resolution advisory (RA) issued by atraffic collision and avoidance system (TCAS) of the aircraft;sequentially highlight each proximate aircraft by traversing the orderedsequence with a displayed cursor; select at least one highlightedproximate aircraft; and designate at least one selected proximateaircraft for at least one traffic application.
 2. The system of claim 1,wherein the one or more interactive traffic symbols comprise: at leastone first traffic symbol corresponding to the at least one firstproximate aircraft, each first traffic symbol associated with a relativeposition of the corresponding first proximate aircraft; and at least onesecond traffic symbol corresponding to each parked aircraft, each secondtraffic symbol displayed along an edge of the HUD based on a relativeposition of the corresponding parked aircraft.
 3. The system of claim 1,wherein the HUD is configured to display, proximate to each interactivetraffic symbol, identifying information corresponding to the associatedproximate aircraft.
 4. The system of claim 1, wherein the HUD isconfigured to display, proximate to each interactive traffic symbolcorresponding to a selected proximate aircraft or a designated proximateaircraft, additional traffic information associated with the proximateaircraft, the additional traffic information selected from a groupincluding: relative position information; airspeed information;trajectory information; or spacing information.
 5. The system of claim4, wherein the additional traffic information is presented via a dynamicdisplay.
 6. The system of claim 1, wherein the HUD is configured todisplay the one or more interactive traffic symbols at an increasedprominence based on a higher priority of the corresponding proximateaircraft within the ordered sequence.
 7. The system of claim 1, whereinthe HUD is configured to display the one or more interactive trafficsymbols at a decreased prominence based on a lower priority of thecorresponding proximate aircraft within the ordered sequence.
 8. Thesystem of claim 1, wherein the HUD is configured to superimpose the oneor more interactive traffic symbols over visual content displayed by theHUD. 9-10. (canceled)
 11. The system of claim 1, wherein: the controlprocessors are configured to detect at least one of a data loss and adata fault associated with the traffic information; and the HUD isconfigured to remove the one or more interactive traffic symbolsassociated with the data loss or the data fault.
 12. The system of claim1, wherein: the HUD includes a head-worn display (HWD) worn by theoperator, the FOV of the HUD substantially aligned with an orientationof the operator.